We propose the use of incomplete dot products (IDP) to dynamically adjust thenumber of input channels used in each layer of a convolutional neural networkduring feedforward inference. IDP adds monotonically non-increasingcoefficients, referred to as a "profile", to the channels during training. Theprofile orders the contribution of each channel in non-increasing order. Atinference time, the number of channels used can be dynamically adjusted totrade off accuracy for lowered power consumption and reduced latency byselecting only a beginning subset of channels. This approach allows for asingle network to dynamically scale over a computation range, as opposed totraining and deploying multiple networks to support different levels ofcomputation scaling. Additionally, we extend the notion to multiple profiles,each optimized for some specific range of computation scaling. We presentexperiments on the computation and accuracy trade-offs of IDP for popular imageclassification models and datasets. We demonstrate that, for MNIST andCIFAR-10, IDP reduces computation significantly, e.g., by 75%, withoutsignificantly compromising accuracy. We argue that IDP provides a convenientand effective means for devices to lower computation costs dynamically toreflect the current computation budget of the system. For example, VGG-16 with50% IDP (using only the first 50% of channels) achieves 70% in accuracy on theCIFAR-10 dataset compared to the standard network which achieves only 35%accuracy when using the reduced channel set.
展开▼
